NO-JIRA: test ai-sbom plugin and verify plugin loading - #8614
Conversation
|
Pipeline controller notification For optional jobs, comment This repository is configured in: LGTM mode |
openshift-ci-robot
added
the
jira/valid-reference
|
@bryan-cox: This pull request explicitly references no jira issue. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
openshift-ci
Bot
added
do-not-merge/needs-area
needs-rebase
|
Please specify an area label DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository YAML (base), Central YAML (inherited) Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (1)
Cache: Disabled due to data retention organization setting Knowledge base: Disabled due to data retention organization setting 📝 WalkthroughWalkthroughThis PR updates the .github/workflows/claude-wif-test.yaml workflow to enable the ai-sbom@ai-helpers plugin in the ai-helpers settings and to replace the Claude test command with an AI SBOM prompt (using claude-opus-4-6). The test now tees CLI output to /tmp/claude-output.txt and validates plugin execution by grepping for SBOM-related markers, printing the captured output and failing the step if no markers are found. Sequence Diagram(s)sequenceDiagram
participant ComponentA
participant ComponentB
ComponentA->>ComponentB: observable interaction
Possibly related PRs
Suggested reviewers
Important Pre-merge checks failedPlease resolve all errors before merging. Addressing warnings is optional. ❌ Failed checks (1 error)
✅ Passed checks (10 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: bryan-cox The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
openshift-ci
Bot
added
the
approved
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/claude-wif-test.yaml:
- Around line 68-74: The workflow currently checks out PR-head contents and runs
the Claude invocation (the `claude -p ...` call) after GCP WIF auth, writing raw
model output to /tmp/claude-output.txt and echoing it with tee/cat — replace
that with a trusted fixture or default-branch checkout instead of the PR head,
remove or revoke GCP WIF auth credentials before calling `claude -p`, and stop
piping raw model output to stdout; write output only to a local file (e.g.,
/tmp/claude-output.txt) and use grep -qi on that file, emitting only safe
success/failure messages (no cat/tee of the full file) or redact sensitive lines
if you must print content. Ensure the changes reference the existing `claude -p
"Generate an AI SBOM for this session." --model claude-opus-4-6 --max-turns 1`,
`/tmp/claude-output.txt`, and any checkout step so reviewers can locate and
update the workflow.
- Line 69: The current grep check uses a broad "sbom" token which can match
normal model output; update the grep pattern in the if that scans
/tmp/claude-output.txt to only match the plugin-specific marker(s) you asserted
(e.g., "ai-assisted" or the exact plugin tag "ai-sbom"/"ai.sbom") so the
workflow only succeeds when the AI-SBOM plugin block is present; locate the line
containing grep -qi "ai-assisted\|ai.sbom\|sbom" and replace the pattern to
exclude the bare "sbom" token, keeping only the distinctive marker(s) like
"ai-assisted" (and optionally "ai-sbom" or "ai.sbom") to tighten the success
check.
- Around line 56-59: The workflow currently clones the moving default branch
with git clone (git clone ... ai-helpers -> /tmp/ai-helpers), which allows
unreviewed plugin changes to be pulled; update the checkout to pin to an
immutable commit SHA by checking out a specific commit in /tmp/ai-helpers
immediately after clone (or using a shallow clone of that SHA) so the
marketplace code used for "$HOME/.claude/plugins/known_marketplaces.json" and
"$HOME/.claude/settings.json" is fixed; ensure the commit SHA is represented as
a variable or literal in the workflow and document/update it deliberately when
you want to change versions.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Enterprise
Run ID: 902d8166-5903-4e3c-a5ce-90eda11a7f32
📒 Files selected for processing (1)
.github/workflows/claude-wif-test.yaml
| git clone --depth 1 https://github.com/openshift-eng/ai-helpers.git /tmp/ai-helpers | ||
| mkdir -p "$HOME/.claude/plugins" | ||
| printf '%s\n' '{"enabledPlugins":{"hello-world@ai-helpers":true,"ai-sbom@ai-helpers":true,"jira@ai-helpers":true,"ci@ai-helpers":true}}' > "$HOME/.claude/settings.json" | ||
| printf '%s\n' '{"ai-helpers":{"source":{"source":"directory","path":"/tmp/ai-helpers"},"installLocation":"/tmp/ai-helpers","lastUpdated":"2025-10-27T12:00:00.000Z"}}' > "$HOME/.claude/plugins/known_marketplaces.json" |
There was a problem hiding this comment.
Pin ai-helpers to an immutable commit.
Line 56 clones the marketplace from the moving default branch, so this trusted workflow can start executing new plugin code without any change in this repo. Please pin the checkout to a specific commit SHA and update it deliberately.
Suggested hardening
- git clone --depth 1 https://github.com/openshift-eng/ai-helpers.git /tmp/ai-helpers
+ AI_HELPERS_SHA="<pinned-commit-sha>"
+ git clone https://github.com/openshift-eng/ai-helpers.git /tmp/ai-helpers
+ git -C /tmp/ai-helpers checkout --detach "$AI_HELPERS_SHA"🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/claude-wif-test.yaml around lines 56 - 59, The workflow
currently clones the moving default branch with git clone (git clone ...
ai-helpers -> /tmp/ai-helpers), which allows unreviewed plugin changes to be
pulled; update the checkout to pin to an immutable commit SHA by checking out a
specific commit in /tmp/ai-helpers immediately after clone (or using a shallow
clone of that SHA) so the marketplace code used for
"$HOME/.claude/plugins/known_marketplaces.json" and
"$HOME/.claude/settings.json" is fixed; ensure the commit SHA is represented as
a variable or literal in the workflow and document/update it deliberately when
you want to change versions.
| claude -p "Generate an AI SBOM for this session." --model claude-opus-4-6 --max-turns 1 | tee /tmp/claude-output.txt | ||
| if grep -qi "ai-assisted\|ai.sbom\|sbom" /tmp/claude-output.txt; then | ||
| echo "Plugin verified: ai-sbom plugin executed successfully" | ||
| else | ||
| echo "ERROR: ai-sbom plugin output not detected" | ||
| cat /tmp/claude-output.txt | ||
| exit 1 |
There was a problem hiding this comment.
Don't run Claude on PR-head contents with WIF creds in scope.
On issue_comment, this job checks out the PR head and then invokes Claude after GCP WIF auth. That lets a fork PR feed attacker-controlled repository files into an agentic tool while cloud credentials are available, and tee/cat will echo any induced exfiltration into the workflow logs. For this smoke test, use a trusted fixture/default-branch checkout instead, or at minimum remove cloud auth and avoid logging raw model output before invoking Claude.
As per coding guidelines, "No secrets in logs; mask sensitive outputs" and "Agentic CI actions: audit for prompt injection via issue/PR title/body flowing into LLM prompts".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/claude-wif-test.yaml around lines 68 - 74, The workflow
currently checks out PR-head contents and runs the Claude invocation (the
`claude -p ...` call) after GCP WIF auth, writing raw model output to
/tmp/claude-output.txt and echoing it with tee/cat — replace that with a trusted
fixture or default-branch checkout instead of the PR head, remove or revoke GCP
WIF auth credentials before calling `claude -p`, and stop piping raw model
output to stdout; write output only to a local file (e.g.,
/tmp/claude-output.txt) and use grep -qi on that file, emitting only safe
success/failure messages (no cat/tee of the full file) or redact sensitive lines
if you must print content. Ensure the changes reference the existing `claude -p
"Generate an AI SBOM for this session." --model claude-opus-4-6 --max-turns 1`,
`/tmp/claude-output.txt`, and any checkout step so reviewers can locate and
update the workflow.
| claude --version | ||
| claude -p "/hello-world:echo HyperShift" --max-turns 1 | ||
| claude -p "Generate an AI SBOM for this session." --model claude-opus-4-6 --max-turns 1 | tee /tmp/claude-output.txt | ||
| if grep -qi "ai-assisted\|ai.sbom\|sbom" /tmp/claude-output.txt; then |
There was a problem hiding this comment.
Tighten the success check to a plugin-only marker.
Including bare sbom in Line 69 makes this pass on normal model output, so the workflow no longer proves ai-sbom loaded. Match only the distinctive plugin marker you called out in the PR, such as the ai-assisted block.
Suggested fix
- if grep -qi "ai-assisted\|ai.sbom\|sbom" /tmp/claude-output.txt; then
+ if grep -qi "ai-assisted" /tmp/claude-output.txt; then📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if grep -qi "ai-assisted\|ai.sbom\|sbom" /tmp/claude-output.txt; then | |
| if grep -qi "ai-assisted" /tmp/claude-output.txt; then |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/claude-wif-test.yaml at line 69, The current grep check
uses a broad "sbom" token which can match normal model output; update the grep
pattern in the if that scans /tmp/claude-output.txt to only match the
plugin-specific marker(s) you asserted (e.g., "ai-assisted" or the exact plugin
tag "ai-sbom"/"ai.sbom") so the workflow only succeeds when the AI-SBOM plugin
block is present; locate the line containing grep -qi
"ai-assisted\|ai.sbom\|sbom" and replace the pattern to exclude the bare "sbom"
token, keeping only the distinctive marker(s) like "ai-assisted" (and optionally
"ai-sbom" or "ai.sbom") to tighten the success check.
|
Now I have the full root cause. Both PRs modify the exact same lines in Test Failure Analysis CompleteJob Information
Test Failure AnalysisErrorSummaryAll three Prow jobs ( Root CausePR #8614 and the already-merged PR #8611 both originate from the same branch (
Because both PRs make conflicting edits to the same region of the file (the plugins setup step and the test command section), git cannot auto-merge PR #8614's commit ( GitHub correctly reports the PR's mergeable status as Recommendations
Evidence
|
The container: directive requires containerMode on ARC runners which needs anyuid SCC and a long-running process. Replace it by installing Claude and cloning ai-helpers directly in workflow steps. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2a6acff to
b1ddcff
Compare
openshift-ci
Bot
removed
the
needs-rebase
|
/test-wif |
3 participants
Summary
ai-sbomplugin in addition to hello-world, jira, ci--model claude-opus-4-6for the test invocationTest plan
/test-wifon any PR to trigger the workflowai-assistedblock🤖 Generated with Claude Code
Summary by CodeRabbit
Note: These are infrastructure/testing updates with no direct impact on user-facing functionality.